A Comprehensive Study on Data Extraction in Sina Weibo

With the rapid growth of users in social networking services, data is generated in thousands of terabytes every day. Practical frameworks for data extraction from social networking sites have not been well investigated yet. In this paper, a methodology for data extraction with respect to Sina Weibo is discussed. In order to design a proper method for data extraction, the properties of complex networks and the challenges when extracting data from complex networks are discussed first. Then, the reason for choosing Sina Weibo as the data source is given. After that, the methods for data gathering are introduced and the techniques for data sampling and data clean-up are discussed. Over 1 million users and hundreds of millions of social relations between them were extracted from Sina Weibo using the methods proposed in this paper.